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ABSTRACT 

A web survey is a cost-effective and efficient method to use when measuring the characteristics of 
an audience and developing or testing new product concepts. This paper reports on the use of a 
web survey by a start-up media/intemet firm, Farmers’ Almanac TV. The results indicate that 
using email to contact respondents from a client list results in data which are of excellent quality 
and quickly obtained. 

Keywords: internet research, web survey, internet sampling, web opinion survey, web survey methodology 


BACKGROUND 


raditional media firms are realizing that consumer and advertising dollars are shifting to the Internet. 
The continuing growth of online advertising, with a 38% increase in first quarter 2006 revenue alone, 
is seen by many as a clear indication that the medium is reaching and engaging its viewers 
(Interactive Advertising Bureau, 2006). Correspondingly, it is estimated that one third of the monies allocated in the 
U.S. on marketing research surveys in the near future will be spent on web-based/online surveys (Johnson, 2006). 



Over two-thirds of Americans 15 years and older use the internet (Comley, 2003; Johnson, 2006) and 
almost 60% of adults in Great Britain report having used the internet (Duffy, Smith, Terhanian, & Bremer, 2005). 
The other top nations for country of origin of web sites in 2002 were Germany, Japan, and Canada (Foy, 2004). 
Clearly, the potential that the worldwide web offers profit and not-for-profit organizations cannot be overstated. 
Web-based (internet-based) surveys are increasingly used by both practitioners and academics. The recent 
proliferation of low-cost, proprietary software programs (e.g., ZipSurvey), enable even the novice researcher to 
design and administer surveys (Hanna, Weinberg, Dant, & Berger, 2005). 


Basically, media businesses have two products to sell: content (to readers and viewers) and audience (to 
advertisers). The demand for new content has encouraged the growth of new programming and innovative mixtures 
of television and Internet content. Farmers’ Almanac TV is a start-up firm that seeks to provide highly entertaining 
information and stories about gardening, weather, and other nature-based topics. As such, it delivers its own version 
of “rural” and “rural want-to-be” users/viewers to its advertisers. But how does a struggling start-up company find 
out, as inexpensively as possible, what content to sell and what the demographics of their audience are? 


After a brief literature review on web survey methodology, this paper will describe the processes 
undertaken by this start-up media business to identify desired content and characteristics of its audience using 
inexpensive, proprietary online survey software. We will describe the hybrid survey methodology employed and 
explain how the information gleaned was used to enhance product offerings and understanding of audience 
characteristics. The authors hope that this case will serve as a teaching tool for both marketing research and 
entrepreneurship professors as well as provide useful information to small businesses. 


The number of research studies on web survey methodology is enormous (Porter & Whitcomb, 2003), but 
the results are often conflicting because much use of the Internet for research purposes occurs in the absence of 
empirically established methodological guidelines that ensure collection of the best possible information. As has 
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been noted by Duffy et al., (2005), studies outlining the advantages and disadvantages of web-based research often 
fail to distinguish between the types of online research and sampling frames utilized. Many of the disadvantages, 
such as lower response rates (Healey, Macpherson & Kuijten, 2005; Roster, Rogers, Albaum & Klein, 2004), 
sampling and population bias (Roster et al., 2004; Shannon & Bradshaw, 2002), and confidentiality (Shannon & 
Bradshaw, 2002) are minimized when known customers/client email contact lists are used (Howell & Smith 2004), 
as is the case for Farmers’ Almanac TV. 

The advantages of online surveys include speed of both administration and turnaround (Healey et al., 2005; 
Johnson, 2006), reduced costs (Healey et al., 2005; Roster et al., 2004; Shannon & Bradshaw, 2002), and reduced 
interviewer error (Dillman, 2000; McCullough, 1998). Hanna et al. (2005) found that respondents to online surveys 
were more thoughtful and, hence, more likely to self-disclose; they tend to provide longer open-ended responses 
(Schaefer & Dillman, 1998) and they may be more candid (Meinert et al., 1998; Truell, 2003). There can also be a 
much higher response rate. Howell and Smith (2004) used a Google-supplied list for their research and obtained a 99 
percent response rate. Online research also eliminates coding and tabulating of the responses because they can be 
directly entered onto a spread sheet when the survey is taken, thus eliminating clerical errors. Interviewer bias 
present in telephone surveys is also eliminated. 

METHODOLOGY 

Unlike the large body of literature on methodologies for mail and telephone survey contact, there is little 
research describing methodologies for collecting data online. Farmers’ Almanac TV used a hybrid survey 
methodology. They sent an email to their client email list with a request to visit a link provided. There, clients 
completed a brief questionnaire. The sample universe consisted of individuals who had signed up to receive a free 
newsletter from FarmersAlmanacTV.com. A random sample of these email addresses was contacted and invited to 
complete the questionnaire. 

Farmers’ Almanac TV designed a questionnaire using ZipSurvey and hosted it on the www.zipsurvey.com 
website (Table 1). The ZipSurvey instrument employs a variety of different question formats to 1) provide feedback 
about the content of the website so that improvements could be made, which would be beneficial to the user and 2) 
identify some characteristics of the target audience that could be used to attract potential sponsors/advertisers for 
both the television program and the website. A third objective was to test the web-based survey methodology itself 
to determine if it should be used in future studies. 

Two samples were selected, one for a pretest and one for the final survey. The pretest sample was obtained 
by selecting a random sample of 2,000 email addresses from the opt-in data base, as was the post-test sample of 
6,000. No changes were made to the instrument as a result of the pretest, so the final sample size was 8,000. 

RESULTS 

The most troublesome aspect of web-based surveys is the sampling methodology employed. Sample 
selection is important since researchers and decision makers usually seek to generalize findings to the population 
from which the sample was selected. Broad-based studies inviting responses in an open forum (i.e., Yahoo or 
Google with banner advertisements inviting participation) are typically criticized as unrepresentative of the general 
population (Sheehan, 2001). However, sampling from a “closed” population is seen as crucial to successful 
sampling because every member of the list is a member of the target audience. With this type of “closed” 
population, a census and both probability and non-probability sampling approachs for selecting participants can be 
used (Sheehan, 2001). 

An example of a closed population is an organizational list, such as members of a faculty, an employee list, 
or a club’s list of members. Shannon and Bradshaw (2002) indicate that to successfully implement electronic 
surveys, survey professionals usually draw samples from organizational lists containing email addresses. This has 
the added benefit of assuring that the recipient views the topic as salient, thus increasing the response rate (Sheehan 
& McMillan, 1999). Further, it allows for some degree of personalization, which helps maximize response 
(Heerewegh 2005). 
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Table 1: Farmers’ Almanac TV Questionnaire on ZipSurvey.com 


Question 

Response Choices 

1. What types of stories would you like to see more 
of on the Farmers’ Almanac TV web site 
(FarmersAlmanacTV.com)? 

How to’s about gardening, weather farming, natural cures 

Feature stories about the environment, green movement, organic foods, solar 
energy, alternative fuels, etc. 

Product profiles for environmentally friendly tools 

Personalized information (based on geography) for best days for planting, 
canning, fishing, etc. 

Traditional Farmers’ Almanac information like “Wit & Wisdom,” Recipes, 
Weather Forecasts 

Other (specify) 

2. How would you like to view these stories? 

As text 

As video 

As text and video 

Other (specify) 

3. What types of stories/information would you like 
to read/view in the Farmers’ Almanac TV 
newsletter? 

Exclusive Farmers’ Almanac Recipes 

Exclusive coupons to use at the FarmersAlmanacTV.com store 

Short videos about gardening, farming, natural cures and more that are not 
available on the Farmers' Almanac TV web site? 

Updates abut the television show 

Best Days information 

Wit & Wisdom (that is not published on the Farmers’ Almanac TV web site) 

Product information from gardening and farming vendors, solar energy 
vendors, and others. 

Long-range weather forecasts not available on Farmers' Almanac TV Web 
site. 

Other (specify) 

4. Where do you live? 

Major city 

Small-to-medium sized city 

Suburb 

Farm 

Other (specify) [Most of these responses were rural]. 

5. Gardening is: 

A hobby 

My job 

My job and my hobby 

Other (specify) 

6. Farming is: 

A hobby 


My job 


My job and my hobby 


Other (specify) 

7. What would you change about the Farmers’ 
Almanac TV newsletter? 

Open-end Response 

8. Have you ever watched Farmers’ Almanac TV 
on public television? 

Yes 

No 

9. What did you like about it? 

Open-end Response 

10. What did you not like about it? 

Open-end Response 

11. Which types of web sites do you regularly visit? 

Gardening web sites 

Health or natural cures web sites 

Agriculture web sties 

Cooking web sites 

Alternative energy web sites 

Environmentally safe product web sites 

Weather web sites 

Other (specify) 

12. Have you ever shopped at the 
FarmersAlmanacTV.com store? 

Yes 

No 

13. Why not? 

I never buy anything online. 

I didn’t see anything that interested me. 

I didn’t know it existed 

Other (specify) 
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Similar to other studies, Farmers’ Almanac TV received immediate responses to its online survey (Weible 
& Wallace, 1998; Schuldt & Trotten, 1994). Unfortunately, the Farmers’Almanac executives had signed up for a 
maximum of 1000 responses, and 800 were received in the first 24 hours. Obviously, the response rate would have 
been higher if executives at Farmers’ Almanac TV had been willing to pay for more responses, and it must be 
recognized that some would-be respondents were excluded. However, management was satisfied with the 
information received and, since each sample group provided similar responses, did not feel it necessary to acquire 
more responses. Once the respondents completed the survey, emails thanking them for their participation were sent. 

The responses were collected and a simple report showing the frequency distribution of the responses as 
well as a list of answers to open-ended questions was available to the executives within minutes of the decision to 
close the survey. ZipSurvey prepares a summary of the findings at any time during the administration of the survey. 
Further analysis could be conducted with data analysis packages, such as SPSS, since the data could be exported in a 
variety of different formats. Although the authors were not given permission to publish the survey results, the results 
were most helpful to Farmers’ Almanac TV and much was learned about the website user and the site’s content. 

Results of this web-based survey were used to completely redesign the website to provide more links 
between web and TV content. The site was changed from 95% video content to 50% video and 50% text content. 
Topics and titles were also changed to address the specific interests of this website audience, including stories about 
weather, gardening, natural cures, astronomy, and cooking. Comments from the survey suggested a desire for faster- 
changing content, so the website is now updated daily and contains content from partner sites so that it is much 
richer in text and video content. The redesigned website includes a more “hip”, or contemporary, blog to address the 
interests of younger respondents. Figures 1 and 2 show the pre-survey web page and post-survey page. 

The Farmers’ Almanac TV website now includes an online store for retail purchases. Since over half the 
survey respondents did not know about the store, the redesigned website has an easy-to-identify call out for the 
online store on every page. Retail sales have improved. 

Figure 1: Pre-Survey Screen Shot 
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Figure 2: Post-Survey Screen Shot 
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IMPLICATIONS 

A high ranking executive at a global research firm predicts that online panels will be “the basis for the 
majority of ad-hoc quantitative research around the developed world within the next two to three years” (Johnson, 
2006, p.23). There are the obvious benefits of low cost and speed, and researchers have enumerated a host of 
additional benefits in using web-based surveys. There may be a tendency for respondents to be more candid 
(Dillman, 2000), to be less inhibited, and to provide long, self-disclosing comments (Foy, 2004; Hanna et al., 2005), 
the elimination of interviewer bias and data entry errors, and higher response rates (Howell and Smith, 2004). 

There is still much to be ascertained about online surveys; however, they appear to be here to stay, given 
the movement away from mail surveys due to expense and telephone interviews due to changing technologies 
attractive to younger people. A large number of Generation Y’ers do not own a land-line telephone with a published 
number, and thus would not be included in the sample frame for land-line telephone interviews (Johnson, 2006). 

Management at Farmers’ Almanac TV made significant improvements to the website based on this survey, 
suggesting that organizations can easily garner low cost information from their clients/customers. The net benefits 
appear to be greatest for those organizations whose sampling frame is an existing customer email contact list. 
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